18.05 Lecture 17 
March 16, 2005 



Properties of Expectation. Law of Large Numbers. 

E(Xi + ... + X n ) = EXi + ... + EY„ 

Matching Problem (n envelopes, n letters) 
Expected number of letters in correct envelopes? 
Y - number of matches 

Xi = {1, letter i matches; 0, otherwise}, Y = X\ + ... + X n 

E(Y) = EYi + ... + EY„, but EYj = 1 x P(Y, = 1) + x ¥(Xi = 0) = P(Y, = 1) = ± 
Therefore, expected match = 1: 

E(Y) = nx- = l 
n 

If Xi, ...,X n are independent, then E(Yi x ... x X n ) = EXi x ... x EX n 
As with the sum property, we will prove for two variables: 
EYiY 2 = EX i x EX 2 

joint p.f. or p.d.f: f(xi,x 2 ) = fi(x 1 )f 2 (x 2 ) 

EXiX 2 = J J Xix 2 f(xi,x 2 )dxidx 2 = J J xix 2 f 1 (x 1 )f 2 (x 2 )dxidx 2 = 
= J fi(xi)xidxi J f 2 {x 2 )x 2 dx 2 = EXi x EX 2 

X U X 2 ,X 3 - independent, uniform on [0,1]. Find EY 1 2 (Y 2 - X 3 ) 2 . 
= EY 1 2 E(Y 2 - X 3 ) 2 by independence. 

= EY 2 E(Y| - 2X 2 X 3 + Xl) = EXf (EX 2 + EY 2 - 2EY 2 Y 3 ) 

By independence of Y 2 , Y 3 ; = EY 2 (EY 2 + EY 2 - 2EY 2 EY 3 ) 

EYi = x x Idx = 1/2, EY 2 = x 2 x Idx = 1/3 (same for Y 2 and Y 3 ) 

EY 2 (Y 2 -Y 3 ) 2 = i(i + i-2(i)(i)) = ^ 

For discrete random variables, X takes values 0, 1, 2, 3, ... 

for n = 0, contribution = 0; for n = 1, P(l); for n = 2, 2P(2); for n = 3, 3P(3); ... 

nx) = Hn=inx>n) 

Example: X - number of trials until success. 

F(success) = p 

F( failure) = 1 — p = q 

oo oo 1 

E(Y) = ^P(Y>n) = ^(l- 2 ,)"- 1 = l + q + q 2 + ... = - = - 

Formula based upon reasoning that the hrst n-1 times resulted in failure. 
Much easier than the original formula: 
EZo nV(X = n) = E~ ! n(l - p^p 

Variance: 

Definition: Var(Y) = E(Y - E(Y)) 2 = cr 2 (Y) 
Measure of the deviation from the expectation (mean). 
Var(Y) = J(Y — E(X)) 2 f(x)dx - moment of inertia. 
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<~ ^2(X — center of gravity ) 2 x in. 
Standard Deviation: 



a(X) = /VarpT) 
V&r(aX + b)= a 2 Var(X) 
a{aX + b) = \a\a(X) 

Proof by definition: 

E{(aX + b) - E(aX + b)) 2 = E(al + b - aE(X) - b) 2 = a 2 E(X - E(X)) 2 = a 2 Var(X) 

Property: Var(X) = EX 2 - (E(X)) 2 
Proof: 

Var(X) = E(X - E(X)) 2 = E(X 2 - 2XE(X) + (E(X)) 2 ) = 
EX 2 - 2E(X) x E(X) + (E(X)) 2 = E(X) 2 - (E(X)) 2 

Example: X ~ U[0, 1] 

EX = [ x x Idx = i EX 2 = [ x 2 xldx=\ 
Jo 2 J 3 

v„(x) = I-(i) 2 = i 

If X u - , X n are independent, then Var(Xi + ... + X n ) = Var(Xi) + ... + Var(X n ) 
Proof: 

VarpTi + X 2 ) = E(X 1 + X 2 - E(Xi + X 2 )f = E((Xi - EX ± ) + (X 2 - EX 2 )f = 
= E{X 1 - EXi) 2 + E(X 2 - EX 2 ) 2 + 2E(.\i - EX 1 )(X 2 - EX 2 ) = 



= Var(Xi) + Var(X 2 ) + 2E(Xj - EXi) x E(X 2 - EX 2 ) 
By independence of X\ and X 2 : 

= VarpCx) +VarpC 2 ) 
Property: Var(aiXi + ... + a n X n + b) = afVar(Xi) + ... + a 2 Var(X„) 

Example: Binomial distribution - B(n,p),¥(X = k) = (£)f> fe (l - p) n ~ k 
X = X\ + ... + X n , Xi = {1, Trial i is success ; 0, Trial i is failure.} 
Var(X) = ££=1 Var(Xi) 

VarpQ - EX 2 - (EX t ) 2 ,EX, = l(p) + 0(1 - p) = p; EX 2 = l 2 (p) +Q 2 (l-p)= p. 
Var(X 2 ) - p - p 2 = p(l - p) 
Var(X) = np(l — p) = npq 

Law of Large Numbers: 

X\,X 2 , ...,X n - independent, identically distributed. 

Xl + ...+X n > TOV 

S n = n — > oo Mi 

n 

Take e > - but small, P(|5„ - EXi | > e) -> as n -> oo 
By Chebyshev's Inequality: 
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¥((S n - EX,) 2 > e 2 ) = ¥(Y > M) < —EY = 

\E(S n - EX,) 2 = ±E( Xl + - + Xn - EX,) 2 = 4var(i(Xx + ... + X n )) 
1 „ T , v \ ,r , v \ \ nVurtX,) Var(Xi) 



for large n. 
** End of Lecture 17 
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